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Abstract 


With the rise in technology use in early childhood classrooms, there is need to explore the 
strategies used to evaluate the effectiveness of such technology. Research indicates the 
proliferation of unvetted technology tools on the market and in online open source formats. 
Meaningful technology evaluation needs to be completed before, during, and after 
implementation in the lesson. Using a qualitative research design, data were collected from 
teacher candidates’ post-practicum reflective essays and one-on-one interviews. Findings 
indicate that during practicum a few teacher candidates used sound pedagogical strategies to 
evaluate the appropriateness and effectiveness of instructional technology. Findings also reveal 
the need for cooperating teachers to have strong technology pedagogical strategies in order to 
help teacher candidates who may be struggling with technology integration and assessment 
strategies. Based on the findings it is recommended that early childhood teacher preparation and 
professional development programs address alternative ways to assess the effectiveness of 
instructional technology tools used in the classrooms. 


Keywords: instructional technology, assessment, early childhood, teacher education, in-service 
teachers. 


1. Introduction 


Technology has transformed pedagogy in early childhood learning environments. 
Most early childhood classrooms today are equipped with some type of technology including but 
not limited to the following; smart boards, projectors, computers, and iPads. Fairly recent 
statistics indicate that more than 95% of teachers in the U.S. have access to computers and 
Internet in the classroom (NCES, 2010). In 2013, NCES reported that 71% of the U.S. population 
3 years and over used the Internet. Inferring from the statistics, one can conclude that more than 
90% of early childhood teachers (K-3" grade) use computer technology with young children today. 
Today’s early childhood technology research, debates and commentaries no longer question (see 
Fool’s gold, A critical look at computers in childhood, 2000) whether technology should be fully 
integrated in childhood education but encourage it. Some encourage it through integration of 
STEM into early learning (Dossani, 2016). Major research funding agencies (such as the Caplan 
Foundation for Early Childhood Education, National Science Foundation (NSF), U.S. Department 
of Education, etc.), support research in early childhood education that seeks to increase knowledge 
and skills on how best to integrate STEM concepts in early learning curriculum. Some of the 
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central questions now are: what type(s) of instructional technology tools are developmentally 
appropriate and effective to use with young children? How do teachers use technology with young 
children to enhance STEM concepts? These questions could be answered when teachers are 
consistent in collecting and analyzing instructional technology assessment data. Of all the 
concepts of STEM, this paper focuses on technology integration and use with young children. 


Early childhood teachers integrate computer technology in the classroom using 
interactive software and media (McManis & Gunnewig, 2012; NAEYC, 2012; Zaranis, 
Kalogiannakis & Papadakis, 2013) to support activities such as; virtual field trips, simulations, 
webquests, and educational games (Jenkinson, 2009; ISTE, 2002; Ntuli & Kyei-Blankson, 2010; 
NAEYC, 2012). Research indicates that in addition to school district-purchased software, early 
childhood teachers spend time sifting through the Internet in search of additional online programs 
(or open source software) that are developmentally appropriate to infuse in their classroom 
activities (Shamburg, 2004). With all the effort that teachers are making to integrate technology, 
little is known about the effectiveness of such technology in young children’s learning process and 
ability to transfer knowledge (Jenkinson, 2009; Ntuli & Kyei-Blankson, 2012; Shields & Behrman, 
2000). The following summary of literature review is on technology integration in the classroom 
and evaluation of instructional technology. 


2. Literature review 
2.1 Technology integration in the classroom 


The bulk of literature related to technology integration in the classroom focuses more 
on the theoretical design of instruction that integrates technology. It is assumed by such 
theoretical designs that evaluation of instructional technology is considered and embedded within 
the instructional planning process. Instructional technology design theory (ITDT) taught during 
teacher preparation follows such assumptions and it is insufficient; most teacher candidates on 
field experience/internship and some novice teachers struggle with application of the theory in 
authentic classrooms because they lack both personal and vicarious experiences (Brown, 2006; 
He & Cooper, 2011; Ertmer, 2005; Judson, 2006; Ma, Williams, Perejean, Lai & Ford, 2008). The 
reality is that most technology courses offered in teacher prep programs lack a meaningful field 
experience component (Bucci & Petrosino et. al., 2004; Ma, et al., 2008). In most cases, teacher 
candidates take field experience courses after completing technology courses which are mostly 
theoretical in nature. In ITDT there are integration models (such as the dynamic instructional 
design (DID) model by Lever-Duffy and McDonald (2011) and the technological pedagogical 
content knowledge (TPACK) model by Mishra and Koehler (2006)) that if applied appropriately 
in authentic classrooms, they result in effective technology integration. Based on the 
aforementioned theoretical frameworks, it is important to study how teacher candidates apply 
theory in the classroom, specifically, how they decide on the types of technology to use and/or how 
they evaluate the effects of such technology before, during, and after lesson implementation. Such 
feedback is important in order to evaluate the impact of instructional technology models used to 
teach technology integration and assessment/vetting/evaluation of instructional technology tools. 
This study is timely because millions of dollars are being invested in learning with technology in 
P-12 schools to boost STEM education (Amiel & Reeves, 2008; Bohlin, 2002; Chen, 2004; STEM 
for All, 2016); therefore, practical knowledge of how to evaluate the effectiveness of instructional 
technology tools invested in schools is necessary. Without the practical knowledge, lot of STEM 
funding and time will be wasted on tools that are not effective. 
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2.2 Evaluation of instructional technology 


The issue of evaluating the effectiveness of educational technology tools has been 
raised mostly by researchers beyond the early childhood discipline (Karolcik, Cipkova & Hrusecky, 
2015; Jenkinson, 2009; Robson & Schraw, 2008). Karolcik et al. (2015: 243) note that “despite 
the fact that digital technologies are more and more used in the learning and educational process, 
there is still lack of professional evaluation tools capable of assessing the quality of digital teaching 
aids in a comprehensive and objective manner”. Jenkinson (2009: 263) argued that the current 
ways of evaluating the efficacy of educational technology are failing to “capture complex 
interactions that occur between the learner and the object”. Robson and Schraw (2008) note that 
current studies that attempt to measure the effectiveness of educational technology report varied 
results. Some of the empirical research that supports the use of certain types of technology in 
learning is not founded on good research design and the results are flawed and biased. Though 
some of the results from such research are generalizable, the studies that led to the results have 
not been replicated to find out if the findings are reliable. Along the same lines, Reeves (2007: 
274) argued that many of such studies are “one of quasi-experimental studies that are not linked 
to any particular research agenda”. Chingos and Grover (2012) indicated that determining the 
effectiveness of any type of instructional materials through large-scale randomized experiments is 
rare because it is expensive and time consuming. In addition, they (ibidem: 6) argued that “many 
instructional materials have not been evaluated at all, much less with studies that produce 
information of use to policymakers and practitioners...this problem ... worsens with the explosion 
of open-source web-based instructional materials”. Most importantly, most of the studies have not 
considered this issue from the perspectives of early childhood teachers and very few of the studies 
have been carried out through the conduct of qualitative research. This article is based on the 
qualitative research conducted with pre-service early childhood teachers on how they evaluate the 
technology tools they use at K-3" grade level during field experiences. The study sought to find 
out the ability of teacher candidates to apply theory into practice, specifically, the methods and 
process utilized in the evaluation of instructional technology tools before, during, and after 
implementation with students. 


2.3 Research question 


The overarching question that guided the study was: What strategies do early 
childhood teacher candidates use during internship to evaluate the effectiveness of the 
instructional technology materials before, during, and after use in their classrooms? 


2.4 Significance of the study 


This study is essential in that it contributes to the literature on teacher evaluation of 
technology tools by exploring this issue from the early childhood teacher candidates’ perspective. 
It is important that teacher candidates learn from cooperating teachers alternative ways in which 
they might evaluate technology tools and programs, especially, in regards to open source web- 
based instructional materials that have so much to offer in the instruction and learning process. 
In addition, this study may help early childhood teacher preparation and technology professional 
development programs to reflect on how they infuse evaluation methods of instructional 
technology in the curriculum. 
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3. Method 
3.1 Research design, context, and participant selection 


In this study, a purposeful qualitative study which used typical sampling was 
employed to explore the strategies teacher candidates use to evaluate the effectiveness of 
instructional technology during internship (or field experience). Typical sampling in a purposeful 
study involves the selection of participants that best represent the population and the 
phenomenon under study (Edmonson & Irby, 2008; Merriam, 2009). It is important to note that 
early childhood refers to children from birth through age 8 (NAEYC, 2012). This study 
purposefully selected teacher candidates preparing to teach 5-8 year olds (K-3" grade). 
Participants of the study included fifteen K-3 teacher candidates (three kindergarten, four ist 
grade, three 2™ grade, and five 3" grade teachers) who were enrolled in the college of education 
in a southeast Idaho university and had completed an instructional technology methods course 
and their first field experience. During field experiences, the teacher candidates were paired with 
a mentor teacher (or cooperating teacher). It is important to note that among the participants, 
four (one 1*t grade, one 2™ grade, and two 3" grade candidates) were in a blended early childhood 
program preparing to certify to teach in both general education and special education classrooms. 
The four candidates were placed in special education classrooms. The assumption in this study 
was that by collecting data from teacher candidates who were supervised by cooperating teachers 
(CTs), somehow, the study would capture what the pre-service teachers observed from CTs. 
(Informed consent was obtained from all participants of this study). Participation in the field 
experience required that the teacher candidates spent a total of 150 hours in the classroom. The 
candidate was to help with designing and planning for technology integrated lessons and activities, 
and implement at least six activities while being monitored by the CT (two of the six activities were 
formally evaluated by university supervisors). The field experience internship provided the 
teacher candidates with an opportunity to apply new technologies and technology integration 
methods learned from the instructional technology course into a real-life classroom, and to 
observe and learn how CTs integrate and evaluate the effectiveness of such technology tools. 


3.2 Summary of technology evaluation process 


Teacher candidates were expected to apply what they learned in instructional 
technology courses. There are processes that teachers should take before they integrate technology 
in the classroom. Table 1 summarizes the processes expected from early childhood teachers to 
ensure that they collect functional data about the appropriateness and effectiveness of tools they 
use with young children. The summary is developed from extensive research-based literature 
review that focuses on developmentally appropriate technology and early childhood technology 
evaluation instruments (Children’s Technology Software Review, 2014; NAEYC, 2012; Haugland, 
2005; Haughland & Wright, 1997; Haugland & Shade, 1994; Ntuli & Kyei-Blankson, 2012; Ntuli & 
Kyei-Blankson, 2013; Wardle, 2002). Information in Table 1 was used to develop the coding 
instrument to be discussed under data collection and analysis. 
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Table 1. Summary of the processes to be completed, 
before, during, and after technology implementation 








Criteria Processes 
Before Discuss with other teachers in the building the technology you want to 
Implementation implement or technology used by other (question what makes it 


developmentally appropriate or effective). 


Search for information about the tool on the Internet (specifically how to 
operate the tool with young children, whether the tool is developmentally 
appropriate* and could be customized). 


Read reviews online about how other teachers have used the tool (write 
down positives and negatives) 


Read peer-reviewed practitioner journals on the use of such technology with 
young children. 


Reflect on how to minimize the negatives if you were to use it in your 
classroom context. 


If the negatives can be minimized, plan for integration of the tool using an 
instructional design model. 


Develop an observational tool that will allow documentation of information 
about the tool as students use. (Include a checklist with desired behaviors or 
skills that students should be able to attain as they use the tool). 





During Monitor the students by moving around and observe how they use the tool. 


Impl i ; 
aaa Use the observational tool to document what you see and hear. (The 


observational tool can include a checklist with pre-determined desired 
behaviors or skills that the teacher wants the student to attain as a result of 
using the tool, on the same observational tool, space for open-ended 
comments can be provided to document what the teacher sees/hears). 


As you move round, ask students if the tool is helping them to complete their 
task with ease or not. 








After Review students’ final products. 

Implementation ; ; ; ; 
Sea Review students’ grades before and after implementation of the tool 

Interview students about the tool in groups and individually. 


Review students’ journal entries about the tool. Guided reflection questions 
are important at elementary level. For instance, I like spellingcity.com 
because... or I had a difficult time using... 


Develop a rubric with students for self-evaluation after using the tool. 








*Developmentally appropriate software is based on the following criteria: The content 
is age appropriate, the vocabulary is age appropriate, the software provides problem 
solving opportunities, the program begins with what children already know and 
gradually introduces the concepts, the software does not provide undesirable 
behaviors, the software encourages active involvement and stimulates the child’s 
interest, the instructions are easy to follow, the program is easy to navigate and allows 
children to use the program independently, and the software allows children to make 
changes in the environment without receiving threatening feedback. The described 
criteria align with official description of age appropriate materials (NAEYC, 1996) and 
technology and media for young children (NAEYC, 2012). 
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3.3 Data collection and data analysis 


Qualitative data were collected in two phases. In the first phase, data were collected 
from the teacher candidates’ field experience reflective essays. Specifically, the researcher used a 
section of the reflective essays which required candidates to address the assessment/evaluation of 
instructional technology materials before, during, and after implementation. In the first phase 
data were coded and themes were generated and organized into categories and subcategories 
(Saldana, 2009). It is important to note that information in Table 1 was used to further organize 
themes into categories and subcategories. In the second phase, one-on-one semi-structured 
interviews were conducted to clarify data collected from reflective essays. The interviews with 
teacher candidates lasted approximately forty-five minutes each. Probing questions were used to 
acquire an in-depth understanding of the phenomenon under study. Data from the interviews 
were coded and analyzed for themes and patterns using open coding (Creswell, 2011; Saldana, 
20090). 


3.4 Establishing credibility and trustworthiness of data collected 


To establish credibility and trustworthiness of qualitative data, the researcher used 
“triangulation” of data and “member checking” (Edmonson & Irby, 2008). Triangulation is a 
technique where the researcher engages in cross checking of data sources, and interpretation 
(Kreftings, 1991). In this study, triangulation involved cross checking data from the reflective 
essays, and interview data. Member checking involves giving the participants data to review for 
accuracy and check for inconsistencies (Edmonson & Irby, 2008; Kreftings, 1991; Lincon & Cuba, 
1985). In this study the participants reviewed the transcribed interview data to confirm or 
disconfirm the reliability of the interpretations derived from the qualitative data. The reflective 
essay assignment guidelines were reviewed by a panel of instructional technology instructors to 
ensure content validity, and alignment with the program standards and ISTE technology 
standards for teachers. The interview protocol was reviewed by subject matter experts to ensure 
reliability and validity of the study (Cuba, 1981). 


4. Findings 


Data from the study were coded and used to answer the research question: What 
strategies do early childhood teachers use to assess the effectiveness of the instructional 
technology materials/tools before, during, and after instruction? Table 2 summarizes the major 
findings to be discussed in detail. 


Table 2. Summary of technology evaluation strategies 
before, during, and after implementation 











Criteria: Kinder- First Second Third 5 
: Evaluation 
Evaluation of garten grade grade grade Stiateesuaned 
instructional materials (n=3) (n=4) (n=3) (m=5) BY 
¢ Reflective ~ = - No information was 
5 z Essays provided 
RQ deo 
mn 5 Interviews confirmed 
B =| no teacher candidates 
+ = Interviews x x x x took part in 
a : 
me evaluation before 
7 implementation 
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V - indicates use of some evaluation strategy; x - no evaluation strategy reported; G - 
general education classroom; S - special education classroom. 


4.1 How do you evaluate the appropriateness and 
effectiveness of technology before integration in the lesson? 


The findings from the reflective essays show that irrespective of the grade level, none 
of the teacher candidates were involved in evaluating the technology tools prior to it being 
incorporated in the lesson. In the reflective prompt they were asked to explain how they evaluate 
technology tools before integration in the lesson. The following excerpts come from teacher 
candidates’ reflective essays: 


Excerpt 1 - Kindergarten teacher candidate: “It was my assumption that the 
technology that was used in the classroom was evaluated by the CT [cooperating 
teacher] during the planning process. I did not get to use any of the tools that we 
compiled during in our course [instructional technology course]. The CT was not sure 
if my tools would be effective, she [CT] did not have time to review them. Because of 
that I used what she planned for”. 


Excerpt 2 - Third grade teacher candidate: “The technology that we used was 
recommended by other early childhood teachers in the same grade level ... if the 
technology was a success in their class it meant we could use it. Teachers in our 
building usually share tools that they evaluated for appropriateness”. 


It was surprising to read Excerpt 1, because all teacher candidates were expected to 
participate in the planning process. A follow up in the interview revealed that the teacher 
candidates had minimal participation in the planning process (except for a few lessons that the 
candidate actually implemented) as that was completed during the weekends or after hours by the 
CT. 


One teacher candidate said: 


“ 


.. most of the time I spent observing ... she [CT] planned the lessons during the 
weekends ... sometimes she worked on her lesson plans after school when I had to be 
back at ISU [college] for my evening classes ... for those lesson plans that I designed 
and implemented, ... I honestly forgot to use the evaluation rubric that you gave us 
in EDUC 3311 [instruction technology course]. 
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Another candidate said: 


“IT planned only for the lessons that were formally observed by the university 
supervisor ... yes, I could have Googled and read what other teachers say about the 
tool but that skipped my mind”. 


Excerpt 2 indicates that the teacher candidate believed an evaluation was completed 
by another teacher before implementation; therefore, the tool is appropriate for use in the same 
grade level. This is contrary to early childhood best practices which advocate for the need to engage 
in reevaluation of instructional materials by individual teachers (Aldridge & Goldman, 2007; 
Coople & Bredekamp, 2009; NAEYC, 2009). It is important to note that being developmentally 
appropriate does not imply that the tool is effective. The process of reevaluating instructional 
materials ensures that materials are adapted to meet the needs of individual students in a 
particular classroom context. All learning is situated, therefore, what works for one teacher may 
not work for another depending on the needs of the students. 


4.2 How do you assess the effectiveness of technology during lesson implementation? 


Findings from reflective essays corroborated by teacher candidate interviews show 
that, typically teacher evaluations during implementation were in the form of observations. 


In a follow up interview, one first grade teacher candidate said: 


“As students worked on the computers, I moved around monitoring if they were able 
to play the game ... sometimes, I played the game to demonstrate how they should do 
it ...”. 


One kindergarten teacher candidate said: 


“... at the technology center there is always someone to help monitoring the kids to 
complete the task ... we were three in the classroom [the CT, teacher candidate, and 
an aid)”. 


Though most teacher candidates indicated that they used observations, third grade 
teacher candidates and two special education candidates in 1t and 2"4 grade indicated that they 
documented their observation data using checklists and anecdotal notes. 


A 3" grade teacher candidate in a special education classroom said: 


“We [the CT and teacher candidate] planned ahead and wrote the skills on a checklist 
that students should be able to meet when they play the game ... here is a checklist 
from my lessons [displaying a sample checklist (using an Ipad) from her technology 
integration e-portfolio]. This is very effective because when we looked at the checklist 
information [data], say over a period of two weeks, we were able to tell if the game 
is working [effectively] for the kids or not”. 


A 2"4 grade special education candidate said: 


“My CT advised me to write anecdotal notes as I moved around observing the 
students. At times we needed to reflect on what we saw students doing or the 
questions that students asked as they worked on the computer. If we did not write 
notes we wouldn’t remember exactly the problems we saw”. 


Looking at the fact that all those placed in special education classrooms used some 
form of documentation of what they observed leads one to falsely believe that special education 
teachers are expected to track their students’ growth more than general education teachers. 
Anecdotal records and checklists are highly encouraged at early childhood level when collecting 
data through observations (McAfee & Bodrova, 2006; McDevitt & Ormorod, 2013) because they 
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allow the teacher to have data for reflective thinking and decision making on whether the 
technology tool is effective or not. 


4.3 How do you evaluate the effectiveness 
of technology after lesson Implementation? 


Findings show that evaluation of the effectiveness of technology after lesson 
implementation involved reviewing and grading students’ assignments (or products) to make 
decisions concerning the effectiveness of the technology used. 


In the reflective essays, one kindergarten teacher candidate wrote: 


“after grading I check to see if students’ scores are high. If the performance is high 
it means the technology is working.” 


A first grade teacher candidate wrote: 


“...1t depends on the quality of what the students are able to produce using the 
technology ...”. 


Relying on such strategies alone is limited because teachers are not able to account 
with certainty if the objectives were met as a result of using the technology. It could be that 
objectives were met due to other instructional materials that supplement the instructional 
technology materials used in the class. 


Interviews also provided another dimension that was not mentioned by teacher 
candidates in K-2 grade level. Teacher candidates at third grade general classrooms indicated that 
they interviewed their own students to learn the extent to which students thought the technology 
was effective. 


One teacher candidate said: 


“After using technology ... I usually have a one-on-one interview with students ... I 
sample students ... I can’t interview all the students...only the high achieving, the 
mediocre, and the low achieving. I ask questions such as - was the game [technology 
tool] helpful? Did you learn anything [new skills] as a result of playing the game? Tell 
me three things [skills] you learned. Were you able to finish your task? Questions vary 
depending on the tool that students used”. 


Another teacher candidate said: 


“My question after the lesson is-will you use it [the tool] again? If not why ... and if 
yes why? This question is powerful because I get to know what makes the technology 
effective. If I hear the same [negative] answer from more than half of the students ... 
that tool is out ... I try another one”. 


Interviewing or questioning students to determine the effectiveness of the technology 
tools in the classroom is one way that is highly recommended (Robyler & Doering, 2013). It is 
important to introduce higher-order questioning from kindergarten so that young children 
develop critical thinking skills that help them make good and appropriate choices. Questioning 
students about the technology tools should not only come at the end of the lesson; teachers should 
ask questions about how the tool is working during the lesson (Lever-Duffy & McDonald, 2011). 
Lever-Duffy and McDonald emphasize the need for continuous feedback from students when 
teaching with technology. This helps with the overall feedback required to make decisions on 
whether to continue integrating that specific technology tool. In some cases, teachers may decide 
to continue with the integration, however, with the use of scaffolds depending on what the student 
interview data would have suggested. 
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Assessment of instructional materials after the lesson should provide a holistic picture 
of the effectiveness of the instructional materials. In addition to what the candidates mentioned, 
teachers may use alternative strategies such developing rubrics and electronic portfolios (Barret, 
2001; Roblyer & Doering, 2012). Teachers need to be encouraged to develop technology rubrics 
that may be used by both the students and the teachers to evaluate the effectiveness of the 
technology (Ntuli & Kyei-Blankson, 2013). Electronic portfolios where students’ work is collected 
over time should include artifacts such as reflective notes on instructional technology tools that 
helped the students to accomplish the task. With young children the teacher can use guided 
reflective prompts (such as, “I like using ... to learn my letter sounds or the program ... was helpful 
in learning about fractions”). Such kind of portfolio artifacts helps the teacher when reflecting on 
the effectiveness of technology tools integrated in the classroom over time. 


4.4 Unique findings 


One reflective essay documented one kindergarten special education teacher’s way of 
assessing the effectiveness of technology during and after the lesson. The teacher candidate 
described how the CT adapted concurrent time series probe approach (CTSPA) which has been 
found to help teachers with technology outcomes documentation (Parette, Blum, & Boeckmann, 
2009; Smith, 2000). The CTSPA has been used in documenting the effectiveness of assistive 
technology and it involves the teacher in collecting performance measures of a child completing a 
specific activity; both with and without technology over a period of time, with the teacher making 
a decision about a reasonable length of time to collect the data (Edyburn, 2002). The candidate 
observed the CT collecting student performance data with and without technology for a month to 
find out the difference (increase or decrease) in the number of students meeting objectives. 
Collection of authentic assessment data about the tool and student performance over a period of 
time for decision making is highly recommended in early childhood education (Johanson, Bell, & 
Daytner, 2008; McAfee, Leong & Bodrova, 2006). In this case, not only did the candidate learn 
about and evaluation strategy but the importance of having a data storage system in place to easily 
store and analyze the effectiveness of the instructional technology tools. Such kind of unique 
experiences during practicum is enriching to teacher candidates. 


Another unique strategy emerged from interviewing a third grade teacher candidate. 
The candidate indicated that they invented the use of the red-cards-up strategy with students. The 
candidate described the red-cards-up as a technique where students are required to individually 
raise a red card during the lesson as a way of alerting the teacher when they need scaffolding. The 
more the teacher has red cards up in a technology-integrated activity; the more likely it is that the 
technology tool is not effective. This strategy has a potential to be effective because of the notes in 
front (“I will use the tool again”) and back (“I will not use the tool again”). If one uses the technique 
appropriately and consistently, they may be able to gather effective assessment data that measures 
the effectiveness of instructional technology materials. The notes on the cards play an important 
role in helping young students to decide if they will use the tool again. The teacher collects the 
cards in two piles at the end of the lesson (organized by students’ choice either front or back) for 
further documentation about the tool. 


5. Discussion 


In this study, all teacher candidates did not participate in the evaluation of technology 
tools before implementation. It is troubling considering the fact that teacher candidates learned 
(in technology integration course) about the processes they need to take to ensure the 
appropriateness and effectiveness of the tools they use with young children. In addition, the 
teacher candidates were paired with CTs so that they could learn from them. However, data 
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indicates that some teacher candidates had little interactive planning time with the CTs. This 
defies the main purpose of field experience for teacher candidates; that is to have authentic 
classroom experience with mentorship. Intentional and focused communication between the CT 
and instructional technology instructors is needed to clarify the role of teacher candidates in the 
classroom, and a discussion of the nature of reflective essays or any other artifacts from the field 
experience is necessary. Not taking away the credit from some CTs who had unique strategies that 
they shared with teacher candidates, it is expected that CTs provide more of such opportunities 
for candidates. If the CTs have limited technology pedagogical and assessment knowledge, they 
need to be encouraged to take technology professional development. 


While data from this study did not yield information on how teachers evaluate 
instructional technology prior to the lesson, early childhood education research strongly 
encourage evaluation of technology before implementation to ensure that it is aligned with early 
childhood curriculum and integration methods. Though most teacher candidates used observation 
strategy to evaluate technology during implementation, they did not practice rigor in 
documentation of the observed data, and there is no consistency across grade levels in terms of 
the strategies used. Overall, the study reveals that teacher candidates have limited strategies and 
skills to evaluate technology in real classrooms despite comprehensive preparation during teacher 
training. One teacher said they forgot to use the evaluation instrument that they learned about in 
one of the technology courses. Early childhood teacher training programs need to encourage 
teachers to engage in the process of evaluating instructional technology all the time to ensure that 
instructional materials are developmentally appropriate, and that they are helping diverse 
students to achieve the learning outcomes. The argument this study brings forth is that early 
childhood cooperating teachers need to apply assessment strategies and techniques consistently 
with teacher candidates; the strategies should align with what is advocated by early childhood 
research and best practices when it comes to the evaluation of the effectiveness of early childhood 
instructional technology materials. This should be reinforced in professional development 
programs. The professional development curriculum may infuse alternative evaluation strategies 
such as those presented in Table 1 that are based on extensive review of early childhood 
educational technology and media materials (Buckleitner, 1999; NAEYC, 2012; Haugland, 2005; 
Haughland & Wright, 1997; Haugland & Wright, 1997; Haugland & Shade, 1994; Ntuli & Kyei- 
Blankson, 2012; Ntuli & Kyei-Blankson 2011). 


5.1 Conclusion and recommendations 


Teacher preparation and professional development programs have a task to bring 
awareness to teacher candidates and in-service teachers about the importance of evaluating 
instructional technology materials, before, during, and after technology integration. Even though 
literature reveals how challenging it is to assess the impact of instructional technology materials, 
that should not encourage early childhood teachers to adopt instructional technology materials 
without individually assessing if they are effective enough to meet the diverse needs of the students 
in different classroom contexts. Given the potential that technology has to increase cognitive 
developmental gains in early childhood, and to support a variety of learning styles, empirical 
research that examines alternative strategies currently used to evaluate the effectiveness of early 
childhood instructional technology materials is crucial. Such feedback is not only necessary to 
compile evaluation strategies that work but also to categorize efficient early childhood 
instructional technology tools. Those who make software would be more focused in developing 
functional technology for early childhood education. 
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